Stability Guide Taiwan Airport Native Ip Node Monitoring And Automatic Switching Strategy

1.

overview and objectives

- goal: ensure high availability and low latency experience of native ip nodes in taiwan airports.
- scope: covers computer room servers, vps, edge hosts and public network domain name resolution links.
- key points: active monitoring, fast automatic switching, and collaboration with cdn/ddos protection.
- indicator-driven: decision-making is based on packet loss, delay, jitter, bandwidth utilization and tcp handshake success rate.
- results expectations: from fault discovery to switchover ≤30s, single node failure recovery sla ≥99.95%.

2.

key monitoring indicators and tools

- latency: icmp/http rtt, threshold example: single-hop rtt>80ms or average rtt>60ms triggers an alarm.
-packet loss: threshold example: 3 consecutive detections of packet loss rate >3% are considered unavailable.
- jitter: udp/tcp jitter >20ms affects real-time services and should be switched first.
- success rate (http 2xx/3xx): http 5xx or tcp handshake failure >5% triggers traffic adjustment.
- tool chain: prometheus+alertmanager, zabbix, smokeping, mtr, bgpmon, ping/curl script and grafana visualization.

3.

automatic switching strategies and implementation methods

- dns level: short ttl (30s) + multiple a records to cooperate with health detection, and the authoritative dns will adjust the weight if necessary.
- routing level: use exabgp or bird to do bgp dynamic withdrawal (withdraw) and announcement to achieve route switching (switching time can be within 20–60s).
- local load: keepalived (vrrp) or haproxy+consul implements l3/l4 layer active drift; heartbeat detection interval example: notify script once every 3 seconds, three consecutive failures trigger switching.
- automation: use ansible or salt to deliver fault handling scripts, and use prometheus alarms to trigger webhooks for switching.
- decision rule example: if the ping packet loss for three consecutive times is >3% or the average rtt is >80ms and the http success rate is <95%, bgp withdraw is triggered and the traffic is directed to the backup node/upstream cdn.

4.

ddos and cdn collaborative protection

- protection strategy: enable iptables speed limit + tcp_synlimit on edge nodes, and then use scrubbing or cloud cleaning upstream.
- anycast vs native ip: native ip retains the real source address to facilitate auditing, but in the face of large traffic, it needs to cooperate with anycast/cdn to take over.
- automatic black hole: set the traffic threshold (for example, inbound traffic >800mbps and the number of connections increases >3x), automatically trigger the upstream black hole or transfer to cdn for cleaning.
- cdn back-to-origin health check: cdn should determine the weight of the back-to-origin node based on back-to-origin http and tcp detection, and automatically fall back to the nearest available node when the back-to-origin fails.
- monitoring linkage: prometheus alarm triggers simultaneously notify the firewall device, bgp controller and dns platform, forming an automatic closed loop.

5.

real case: taiwan airport a/b/c/d four-node switching demonstration

- scenario: 4 native ip edge nodes are deployed at taipei/taichung/kaohsiung airport to provide real-time information about the boarding gate.
- observation period: sampling every 15 seconds, summarizing and generating health scores every 1 minute.
- trigger condition: if the health score of a node is lower than 60% for 2 consecutive minutes, it will be removed from the scheduling pool and trigger bgp/dns switching.
- result: a nighttime link failure caused a surge in packet loss at taichung nodes, and the system completed the switchover within 45 seconds without obvious service interruption.
- the following table shows the real-time monitoring data at the moment of a fault (example):

6.

implementation checklist and server configuration example

- recommended server specifications (example): 4 vcpu, 8gb ram, 100gb nvme, 1gbps public network bandwidth, ubuntu 20.04, kernel 5.4+.
- network parameters: set mtu=1500, adjust net.ipv4.tcp_tw_reuse=1, tcp_fin_timeout=30 to deal with concurrent short connections.
- keepalived example snippet: vrrp_interval 3, vrrp_script health_check { script "/usr/local/bin/health.sh" interval 3 weight 2 }.
- health check script example: use curl -ss -m 5 http://127.0.0.1:8080/health || exit 1; measure ping and tcp ports at the same time.
- deployment steps: 1) establish prometheus collection and alarm; 2) configure keepalived/haproxy and bgp controller; 3) practice switching and record rto/rpo, and retest periodically.